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Abstract 

We consider optimal control problem with an integral cost which is a mean of a given 
function. As a particular case, the cost concerned is the Cesaro average. The limit of the value 
with Cesaro mean when the horizon tends to infinity is widely studied in the literature. We 
address the more general question of the existence of a limit when the averaging parameter 
converges, for values defined with means of general types. 

We consider a given function and a family of costs defined as the mean of the function 
with respect to a family of probability measures - the evaluations - on R+. We provide 
conditions on the evaluations in order to obtain the uniform convergence of the associated 
value function (when the parameter of the family converges). 

Our main result gives a necessary and sufficient condition in term of the total variation of 
the family of probability measures on R+. As a byproduct, we obtain the existence of a limit 
value (for general means) for control systems having a compact invariant set and satisfying 
suitable nonexpansive property. 
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1 Introduction 


We consider a control system defined on whose dynamic is given by 

y'{t) = f{y{t),u{t)) ( 1 . 1 ) 

where / : x [/ —>■ and u{-) is a measurable function - called the control - from M_|_ to U 

a fixed metric space. We will make later on assumptions on ra ensuring that for any initial 
condition y(0) = yo, and any measurable control u{-), the equation (II.ip has a unique solution 
t !-)• y{t, u, yo) defined on R+. 

To any pair {yo,u{-)'j, we associate a cost 

+ 00 

9{y{t,u,yo),u{t))d9{t), 

where g : x U ^ M. is Borel measurable bounded and 0 E A(R_|_) is a Borel probability 

measure on 1R+. We called 0 an evaluation throughout the article. 


We will refer to the previously described optimal control problem by the short notation J' = 
{U,g,f). Let 6 € A(R_|_), we define for J' = {U,g,f) the following value function: 


r *+00 


Veiyo) = inf / 
Jo 


( 1 . 2 ) 


9 {yit,u,yo),u{t))d 0 {t), 
where U denotes the set of measurable controls u : [0, Too) ^ U. 

Typical means in the dehnition (jl.2p of the value function are well studied in the literature for 
Cesaro mean: Vf > 0, dt with density s i— >■ /^^(s) = jl[o,t](s), and the t-horizon value is 

1 

\iyo) = inf 7/ 9{y{s,u,yo),u{s))ds 
u(-)eu t Jo 


Abel mean: VA E (0,1], 6x with density s i—>■ fe^^is) = Xe and the A-discounted value is 


r+oo 

Vexiyo) = inf / Xe~^^g{y{s,u,yo),u{s))ds 

u{-)eu Jo 


The limit of the above value functions as t tends to inhnity or as A tends to zero are well 
investigated in the control literature, (cf. [T], [2], |3], H], [6], [7] and the references therein), 
which are often called ergodic control. 

When 9 E A(R_|_) is given, the contribution of the interval [T,+oo) in the mean (II.2p is less 
and less significant as T becomes large. Thus the control problem is essentially interesting only 
on [0,To] for certain To, which we roughly name the "duration" for the problem. In this article, 
we are interested in the long-run property of J , i.e., the asymptotic behavior of the function 
9 Vg when the "duration" of 9 tends to infinity. In the particular examples of Cesaro mean 
and Abel mean, the uniform convergence of as t tends to inhnity and of Vg^ as A tends to 0 
are studied. It is a priori unclear how to dehne the "duration" of a general evaluation 9 over R_|_. 
If one just assumes the expectation of 9 to be large, we can obtain very different value functions, 
as is shown by the following 


2 


Example 1.1 Consider the uncontrolled dynamic y{t) = t, the running cost t i—)• g[t) = 
lu°o_^[2m-i,2r7i](^); sequences of evaluations CLnd {i'^)k>i with densities: f^k = 

fu^ = [ 2 m- 2 , 2 m-i] • Clearly, V^k = 1 and 14fc = 0, VA: > 1. 

For this reason, we introduce an asymptotic regularity condition for evaluations, called the 
long-term condition (LTC for short), to express the "large duration" and the "asymptotic uni¬ 
formity of distributions over M_|_", and we will study the convergence of the value functions along 
a sequence of evaluations satisfying the LTC. 

More precisely, for any s > 0, we define the s-total variation of an evaluation 6 to be the total 
variation between the measure 9 and its s-shift along M_|_: 

TVs{e) = max \e{Q) - 9{Q + s)|. 

QeB(]R+) 

We say that a sequence of evaluations {9^)k>i satishes the LTC if: 

V5 > 0, sup TVsie'^) -^ 0. 

0<s<S k^oo 

The optimal control problem J = {U, g, f) has a general limit value given by some function V* 
defined on if for any sequence {9^)k satisfying the LTC, (V 0 fc(yo))fc converges uniformly to V* 
as k tends to infinity. 

Our main result (Theorem 14.11) states that for any {9^)k satisfying the LTC, {VQk)k converges 
uniformly if and only if the family {Vgk} is totally bounded with respect to the uniform norm. 
Moreover, in this case, the limit is characterized by the following: 

V*{yo)=def sup inf inf / g(y{t + s,u,yo),u{t + s))de{t), Vyo € M'^. (1.3) 

6 »gA(R+) io ^ 

The above function V* naturally appears to be the unique possible long-term value function of 
the control problem. 

As a byproduct of our main result, we obtain the existence of the general limit value for any 
control problem J = {U, g, f) with a running cost g that does not depend on u and with a control 
dynamic (HD which is non-expansive and has a compact invariant set. This can be viewed as a 
generalization of already obtained results in [8] for optimal control with Cesaro mean. 

Existing results in the erdogic control literature are concerned mainly with the convergence of 
the t-horizon Cesaro mean values or the convergence of the A-discounted Abel mean values. To 
the best of the authors’ knowledge, this paper is the first to consider general long-term evaluations 
for optimal control problems. 

Also it is worth pointing out that while many works (including [T], |^, [3], |4], [6], [7]) suppose 
controllability or ergodicity conditions, the present approach does not reply on such conditions. 
This could be understood by the fact that the limit value V* may depend on the initial state yo 
(which does not occur under ergodic or controlability assumptions). 

We also make here a link to the discrete time framework, in which an evaluation 9 = {9m)m>i 
is a probability measure over positive integers N* = N\{0}, and 9^ is the weight for the stage- 
t payoff. The analogue notion of total variation is dehned for any 9 € A(N*): TV(9) = 
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Ylrn=i l^m+1 ~^m| (cf- |12j and [9]). Recently, the existence of the general limit value of dynamic 
optimization problems in several discrete time frameworks has been obtained in and m- 

Our work is partially inspired by |9] and similar tool within the proof appeared in |10| . 

The article is organized as follows. Section [2] contains some preliminary notations and basic 
examples. The long-term condition is introduced and studied in Section [3l Section 0] contains our 
main result and its consequences. We discuss in the end of this section two (counter)examples. 
Section [5] is devoted to the proof of the main result. A weaker notation of LTC is discussed in 
Section 01 


2 Preliminaries 


Consider the optimal control problem J = {U,g,f) described by (ll.ip - (ll.2p . We make the 
following assumptions on g and /: 

the function g : x [/ —>■ R is Borel measurable and bounded; 

the function / : R*^ x [/ —>■ R*^ is Borel measurable, and satisfies: 

(*). 3L > 0,y{y,y) G R2'^,Vm G U,\\f{y,u) - f{y,u)\\ < L\\y-y\\, 

3a>0,V(?/,u) gR'^x U,\\f{y,u)\\ < a(l||y||). 


Under these hypotheses, given any control u(-) in U and any initial starting state yo G R^ (HH) 
has a unique absolutely continuous solution t i— y{t,u,yo) defined on [0,-|-oo). As the running 
cost function s' : R'^ x [/ —>■ R is bounded, we can always assume that 5 ; R'^ x ^ [0,1] after 
some affine transformation. 


Below we introduce several notations. 


0-evaluated cost 'y 0 {yQ,u) Given 6 G A(R_|_) and yo G R*^, the 0-evaluated cost induced by a 
control u{-) G ZY is denoted by: 


l8{yo,u) 



With this notation, the 0-value function in (II. 2 p writes as Vg{yo) = inf^^qg^ 70(2/0) ^)- 
Reachable map Rt For any yo G R'^, the reachable map in R+, 1 1 —)• Rt{yo), is defined as: 

Rtiyo) = |y G R'^|3 u(-) gU : y{t,u,yo) =y|. (2.2) 


Rt{yo) represents the set of states that the dynamic can reach via certain control at time t, start¬ 
ing from the initial state yo at time 0. We write R*(yo) = U*^o-^s(yo) R{yo) = U“o-^s(yo)- 
iZ(yo) is the set of states that can be reached at any finite time starting from yo. 


Image measure and the auxiliary value function Vji^g Given t G R and 9 in A(R+), 
we use Tt ‘^6 to denote the image (push-forward) measure of 9 by the function % : s t-G’s + 1 , i.e., 

TMQ) = 0 {rf\Q)) =9{{Q-t)n R+), VQ G R(R+), 
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where ;S(M+) denotes the set of all Borel subsets in M+. This leads us to write the t-shift 
0 -evaluated cost induced by a control u as follow: 

'lTtte{yo,u) = g(y{s + t,u,yo),u{s + t))de{s), '^t>0. (2.3) 

JlO,+oo) ^ ^ 

Taking on both sides of (I2.3jl the inhmum over u{-) G U and using the notation of reachable map 
Rt, we obtain the t-shift 0-value function 

VTtieiyo)= inf [ givis+ t,u,yo),u{s+ t))d6{s) = _ inf Ve{y). (2.4) 

«(-)eW J[o,+oo) ^ ^ y&Rt{yo) 

s-total variation Given an evaluation 6 , define its s-total variation for each s > 0: 

TVs{e)= sup \e{Q) - e{Q + s)\. ( 2 . 5 ) 

(9g/?(R+) 

Long-term condition (LTC) A sequence of evaluations {0^)k>i satisfies the LTC if: 

VS > 0, TVsid’^) =def sup TVs{9’^) -^ 0. 

0<s<S k^oo 

In this article, we are concerned with the following notation of limit value for optimal 
problems with general means. 

Definition 2.1 Let V be a function defined on M'^. The optimal control problem J admits V as 
the general limit value if: for any sequence of evaluations {9^)k>i satisfying the LTC, for all 
yQ in (f^fc(yo)) converges to V{yQ) as k tends to infinity, and moreover the convergence is 
uniform in yQ. 

Below are some basic examples of optimal control problems in which the general limit value 
exists. 

Example 2.2 y lies in seen as the complex plane, there is no control, and the dynamic is 

given by f{y) = i y, where = —1. We clearly have 

Vek{yo) - -^ — / 5((|yo|e”‘)dt, 

k^oo ZTT Jq 

for any sequence of evaluations {9^)k satisfying the LTC. 


( 2 . 6 ) 

control 


Example 2.3 y lies in the complex plane again, with f{y,u) = i y u, where u ^ U is a given 
bounded subset ofM, and g is any continuous function in y (which thus does not depend on u). 

Example 2.4 f{y,u) = —y + u, where u & U a given bounded subset o/R'^, and g is any 
continuous function in y (which thus does not depend on u). 


We will show later (using Corollary 14.7p that the general limit value exists in Examples 12.31 and 

Eai 
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3 On the long-term condition (LTC) 


In this section, we discuss the LTC. First, we give the following remarks. 

Remark 3.1 (a). By definition, one has 

Vs > 0,Vt > O,V0 G A(M+), TVs+t{0) < TVs{e) + TVfie). 

This implies that {0^)k>i satisfies the LTC if and only ifTVi{9^) - > 0. 

~ k^oo 

(b). If one takes Q = M+ in definition of TVs{0^) for each s > 0 and each k > 1, we deduce that 
if {6^)k>i satisfies the LTC, then 0^([O, s]) -0 for any s > 0. 

~ k^oo 

Remark 3.2 Let 9 he an evaluation absolutely continuous w.r.t. the Lebesgue measure on M+, 
and fg its density. Scheffe Theorem (cf. Theorem 1 in p.2) implies that: 

1*00 

Vs > 0, 2TVs{9) = Is{9) =def / \fe{t + s)- fe{t)\dt. 

Jo 

Thus, if {9^)k>i is « sequence of evaluations with densities {fgk)k>i-' 

(a). {9’^)k>i satisfies the LTC if and only z/supg<g<]^/s(0^) -?■ 0. If moreover, for each 

~ ~ ~ k^oo 

k > 1, 1fgk{t) is non increasing on M_|_, then {9^)k>i satisfies the LTC if and only if'ds > 0, 
^'''([0, s]) = ft=o f8it)dt - /“o fg{t + s)dt - -^ 0. 

AC—>^00 

(h). if {9^)k>i satisfies the LTC, then f^Qtfgk(t)dt — —> oo. Indeed, Chebychev’s inequality 
gives that J^Qtfgk{t)dt > M (l — 0^([O,M])) for all M > 0. 

Here we discuss several cases where the LTC condition is satisfied. 

Example 3.3 (Uniform distributions) Assume that for each k, 9^ is the uniform law over the 
interval [ak,bk], with 0 < < bk- For each k. 


• s ^ dfi . 

Is{9^) = < 

r 2 

^k 

< 6/c O-k 

) 

if 0 < s < Ok 
if Ok < s <bk 




if bk < s 

• s bf^ d }^. 

Is{9^) = < 

( 2s 

^k~^k 

2s 

if 

if 

0 < s < dk 

^k ^ ^ ^ ^k 



fJ'k 


One can check easily that {9^)k satisfies the LTC if and only if bk — ak - > oo. Indeed, by 

k^oo 

Remark \3.2\ (a), it is sufficient to look at Is{9^) for s G [0,1]. 

Example 3.4 (Abel average) Assume that for each k, 9^ has density s fgk{s) = (s), 

with Xk > 0. Since V/c > 1, s i—>■ fgk{s) is non increasing. Remark Ig.^l (a) implies that {9^)k 

satisfies the LTC if and only if: VT > 0, 9^{[0,T]) = Xke~^^^ds = 1 — — - > 0, 

which is again equivalent to Xk -)• 0. 

k—^oo 
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Example 3.5 (Folded normal distributions) Assume that for each k, 6 ^ is the distribution of a 
random variable \X^\, where follows a normal law Af{mk, . The density of 9^ is given by: 


Vi > 0, fQk{t) = 






exp 



Claim 3.1 {9^)k satisfies the LTC if and only if - > oo 


k^oo 


Our argument relies on the following lemma, whose proof is put in the Appendix. Without loss 
of generality, we may assume that mk is non-negative for each k. 

Lemma 1 Let 6 be the distribution of X where |A| follows the normal law M{m, a) with m,a > 
0. There exists some t* G [0,m) such that f'gft) > 0 for any t G (0, i*) and f^^t) < 0 for any 
t G (i*,oo). Moreover, such t* satisfies that: (i*)^ > — cr^. 


Proof of Claim 13.11 We apply Lemma[T]to each evaluation 9k to obtain some G [0, mk) such 
that: f 0 k{-) is increasing on [0,i^) and decreasing on [i^,oo). This enables us to write: 


Vs < i 


— ''fc) 


h{ 0 ^)= ! f 0 k{t)dt+ [ \f 0 k{t + s) - f 0 k{t)\dt + [ f 0 kit)dt 

Jtt-S Jtt-S Jtl 


We deduce then sf 0 k{t)^ — s) < l 0 k{s) < 4s/efc(i^) for s < Assume below i* 

liminffc_).oo > 0, and the analysis is analogue for t* = 0, which we omit here. 

(*). Suppose that cjfc —>■ oo, then 


-def 


fe'^i'tk) — 




exp 



/ 1 ftl + mk' 

-d — 


< 


akV ^ A:->00 


> 0 . 


This implies that for S = i* Al, supo<s <5 Is{G^) -1 0. 


fc—>-oo 


(**). Conversely, suppose that {9^)k satisfies the LTC. Then for any s < i*, Is{9^) thus f 0 k{T(, — s) 
vanishes as k tends to infinity. This implies that either cjfc —00 or {ak)k is bounded and 
{mk — — s))^ —7> 00 . Lemma [T] shows that the specified point for the evaluation 9k satisfies 

>ml- al, thus - {tl + s) <mk-tl< If {(Tk)k is bounded, (m^ - tl)k 

2 

thus {mk)k should tend to infinity, but this leads to a contradiction with mk — D 


Now we link the LTC condition to the discrete time framework. In a discrete time dynamic 
optimization problem, a general evaluation on the payoff stream is a probability distribution over 
N* = N/{0} the set of postive integers. For any = (,^ 1 , ...) in A(N*), its "total variation" 

(0 = Ylm=i l?m+i ~ I® Stage by stage absolute difference between the measure ^ and 
its one-stage "shift" measure = {^ 2 , ■■■)■ (cf- Sorin |12j or Renault |11|). 

When the sequence of evaluations in continuous time admits step functions as densities, this 
link to discrete time framework is much clearer as seen by the following 


Proposition 3.6 Let {9^)k be a sequence of absolutely continuous evaluations in A(M_|_), and 
their densities are given as: Mk > l,f 0 k = Z]m=i where ffi = ) G 

A(N*). Then {9^)k satisfies the LTC if and only ifYlm=i l?m+i “ 


fc—>-oo 


0 . 
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Proof: Fix s G [0,1]. We shall write for each fc, 


w”) = E / 


fekit + s)-fgk{t) 


dt. 


For each m = 1,2,..., we have 


/ 

J [m—l,m) 


fQk{t + S) — f0k{t) 


dt = 


' [m—l,m—s) 
■sm 


fQk{t + s) — fek{t) 


dt+ (f 9 i^(t + s)-fgk{t))dt 


'’|Sm+l Sr) 


As a consequence, 


/.(«‘) = 0 E l«™+i - s E l«™+i - [“■ 


m=l 


m=l 


In view of Remark 13.21 {6^)k satisfies the LTC if and only if X]m=i l?m+i ~ 


fc^OO 


0 . □ 


We end this section by a preliminary lemma, which will be useful in later results. 


Lemma 3.7 Fix any 6 G A(M+) and any t G M+, we have 


/ /i(s)d0(s) — / h{s — t)d6{s] 

'[0,+oo) J[0,+oo) 


< TVt{e) and 


/ /i(s)d0(s) — / /i(s + f)d0(s^ 

'[0,+oo) J[0,+oo) 


< 2TVt{e), 


for any h{ ) G Ad(M+, [0,1]), where A4(M+, [0,1]) = h : M+ [0,1], Borel measurable^. 


Proof: We fix any 6 G A(M_|_) and t G M+. By definition of TsW^ have that for any 
/i(-) gM(]R+,[0,1]): 


/ h{s)d6{s) — / h{s — t)d9{s)= / h{s)d9{s) — / h{s)dT-tW{s) (3-1) 

J[0,+oo) J[t,+oo) J[0,+oo) J[0,+oo) 

and 

f h{s)d9{s) — f h{s + t)d9{s)= f h{s)d9{s) — f h{s)dTt‘^9{s). (3.2) 

J[0,+c>o) J [0,+oa) J[0,+OC}) J[0,+oo) 

Since 'T-t'^9 and 7ttj0 are both Borel measures on M_|_, "9 — T-t^9" and "9 — Tt‘^9" are both signed 
measures. Hahn’s decomposition theorem^ implies that: 


sup 

f h{s)d9{s) - f h{s)dT-tP{s) 

= sup 

9{Q)-T-tmQ) 

/iGA^(K+,[0,l]) 

J[0,+oo) ./[0,+oo) 

Qg/?(R+) 



and 


sup 

[ h{s)d9{s) - [ h{s)dTtms) 

= sup 

9{Q)-rtmQ) 

hex (R+, [0,1]) 

./[0,+oo) ./[0,+oo) 

(9g/?(R+) 



^The first author acknowledges Eilon Solan for the discussion on using Hahn’s decomposition theorem. 























Combining with (|3.1I) - (|3.2I) . we obtain: 


f h{s)d6{s) — [ h{s — t)d0(s) 

< sup 

e{Q)-e{Q + t) 

./[0,+oo) J[t,-eoo) 

QeB{R+) 



TVt{e) 


and 


f h{s)d6{s) — f h{st)d9{s) 

< sup 

9{Q)-0{Q-t) 

■j[0,+oo) 7(0,+oo) 

QeB(R+) 



The proof of the lemma is complete. 


□ 


4 Main Result 


As will be shown in our main result, the function V*{yQ) defined in (|1.3I) characterizes the general 
limit value of the optimal control problem in case of convergence. We first rewrite it as 

W(yo) = sup inf V7^tt/^(7/o) = sup _inf ^(y). 

/iGA(R+) fiGA{R+) y^Rivo) 

We give the following interpretation: consider the auxiliary optimal control problem (game) 
where an adversary of the controller chooses an evaluation /x, and then knowing /x as given, the 
controller chooses an initial state in the reachable set R{yo). The running cost from time t is 
evaluated by /x and V*{yo) is the value of this problem starting from yo- 

Recall that a metric space X is totally bounded if for each e > 0, X can be covered by finitely 
many balls of radius e. 

Theorem 4.1 Let {6^)k>i be a sequence of evaluations satisfying the LTC. Assume 112. il) for 
the optimal control problem J = {U,g,f). Then, 

(i). R* = supfcgNinftgK+ VTt^ek- 

(a). Any accumulation point (for the uniform convergence) of the sequence (Vgfe)^ is equal to 

V*. 

(Hi). The sequence {Vgk)k uniformly converges if and only if the space • ||oo) is totally 

bounded. 


Remark 4.2 Let {0^)k be a sequence of evaluations which contains a subsequence {9^^)k satis¬ 
fying the LTC. Then Part (z) of Theorem \4-l\ still holds true for {6^)k- 


A more precise convergence result is obtained if we suppose that there exists a compact set 
T C which is invariant for the dynamic m, i-e., such that y{t,u,yo) G Y for all u{-) G U, 
t > 0 and yo in Y. 


Definition 4.3 Let V be a function defined on W^. In the optimal control problem J, there is 
the general uniform convergence of the value functions {Vg} to V if: 

Ve > 0, 35 > 0, > 0 s.t. MO G A(M+), with TVs{0) <g, \\Vg — V\\oo < £. 
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Lemma 4.4 Let V be a funetion defined on The optimal control problem J admits V as the 
general limit value if and only if there is the general uniform convergence of the value functions 
{Ve} to V. 


Proof. The general uniform convergence of the value functions {V^} to V implies the existence 
of general limit value given as V. Next we show that the existence of the general limit value 
given as V is sufficient to deduce the general uniform convergence of {Vg} to V. Suppose by 
contradiction that there is no general uniform convergence of {V^} to V, i.e., 

3eo > 0, V5 > 0, > 0, 30^ E A(M+) with TVs{0^) < and \ \Vgk - P||oo > eo, Vfc > 1. 

Let ffo > 0 be fixed as above. We take a vanishing positive sequence and some Sq > 0, then 
there is a sequence of evaluations [9^) with TV - > 0, and liminf^ HV^fe — P|loo > 

k—^oo 

Sq. According to Remark 13.11 fab such {6^)k satisfies the LTC, while (Vgk) does not converges 
uniformly to V*. This is a contradiction. ■ 


Corollary 4.5 Assume h2.1\) for the optimal control problem J = {U,g,f). Suppose that there 
is a compact set T C which is invariant for the dynamic (EIP, and that the family {Vg \ 0 E 
A(M_|_)} is uniformly equicontinuous on Y. Then there is the general uniform convergence of the 
value functions {Vg} to V*. 


Proof: By assumption, the family of value functions {Vg : 6 G A(R_|_)} is both uniformly 
bounded and uniformly equicontinuous on the compact invariant set Y, so we can use Ascoli’s 
theorem to deduce the totally boundedness of the space {{Vg}, || • ||oo)- Theorem 14.II implies that: 
for any {9^)k satisfying the LTC, the corresponding sequence of value functions {Vgk) converges 
uniformly to V* as k tends to infinity. Thus J' has a general limit value given as V*, and 
according to Lemma 14.41 there is the uniform convergence of value functions {Vg} to V*. □ 

We shall give the existence result of the general limit value under sufficient conditions expressed 
directly in terms of properties of the control dynamic and of the running cost g. 

Let us introduce the following non expansive condition (cf. |8]). The control dynamic (jl.ljl 
is non expansive if 

Vyi, 2/2 G sup inf (yi - 7/2, f{yi,a) - f{y 2 , b)) < 0 . 

asubeu \ / 


Definition 4.6 The optimal control problem J = {U,g,f) is called compact non expansive 
if it satisfies the following three conditions: 

(A.l) there is a compact set Y C is the invariant for the dynamic 

{A.2) the running cost function g{-) does not depend on u GU , and is continuous in y G 

(A.3) the control dynamic hl.l\) is non expansive on Y. 


Corollary 4.7 Assume h2. j|) for the optimal control problem J = {U,g,f). Suppose that that 
J is compact non expansive, then the general limit value exists in J and is given as V*. 
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Proof: Under (A.l) and (^.3), Proposition 3.7 in jSj implies that: 


V(yi,y 2 ) G h"^V^6(•) G G U,s.t. Vt > 0, \\y{t,u,yi) - y{t,v,y 2 )\\ < ||yi - y 2 \\- (4.1) 


We claim that the family is uniformly equicontinuous on Y, thus Corollary 14.51 and 

Lemma [4.41 apply. Fix any {yi,y 2 ) G Y^, 6 G A(M_|_), and e > 0. Let u be e-optimal for 14 ( 2 / 1 ): 


L 



14 ( 2 / 1 ) > / 9{y{s,u,yi))d9{s) - e. 

J[0,+oo) ^ ^ 

According to the non expansiye property, there exists v{-) mU as in (14.ip such that 

\\y{s,u,yi) - y{s,v,y 2 )\\ < \\yi - 2/2II, Vs > 0 . 

By definition, 14 ( 2 / 2 ) < /[g ff( 2 /(s, 2 / 2 , u))d6'(s), hence 


(4.2) 


X 


9 {y{s, V, 2/2)) - 9 {y{s, U, 2/1)) d 6 »(s) + e. 


14(2/2) - 14(2/1) < 


[0,+oo) 


Denoting Ug the modulus of continuity of g, we obtain in yiew of (14.2D : 



4[0,+cx)) 


Interchanging 2/1 and 2/2 and taking into account of e > 0 being arbitrary, we deduce that 


(14)6»gA(R+) is uniformly equicontinuous on the inyariant set Y. This finishes the proof. □ 


Remark 4.8 Both Examvle \2.9\ and Example \2.4\ satisfy conditions of Corollary \4. T[ so there is 
general uniform convergence of the value functions { 14 } (the existence of the general limit value). 


Remark 4.9 Our result generalizes Proposition 3.3 in m which proved the uniform convergence 


of the t-horizon values in compact non expansive optimal control problems. 

We end this section by presenting two (counter)examples, showing that the results in Theorem 
O do not hold if some of their conditions is not satisfied. 


The first example is an uncontrolled dynamic. We show that if {9^)k contains no subse¬ 


quence satisfying the LTC, then the result in Part (i) of Theorem 14.11 does not hold, i.e., 
supfc>i inft>o 14^ttefc(2/o) < sup 0 eA(R+) infi>o 14 'ttt 0 ( 2 /o) for some 2/0 (cf. RemarklO]). 

Counter-example 4.10 Consider the uncontrolled dynamic on M.' 2/(0) = 2 /o y'{t) = 

— {y{t) — 1), Vt > 0. The trajectory is then y(t) = 1 + ( 2/0 — l)e“h The running cost func¬ 
tion (/ : M —7- [0,1] is given by: 



if 

if 

if 


0 

y 

1 
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2 / < 0 

0 < 2 / < 1 

y > 1 






We. have that V*{yo) = sup 5 )g^(]g_|_) inftgjR,^ VT4e{yo) = 1, Vt/q £ Indeed, let yo be given and 
fix any e > 0, there is some > 0 such that \y{T) — 1| < e for all T > T^. Take an evaluation 
9 in A(]R_|_) with ^([OjTe]) = 0. This enables us to deduce that: for all t >0, 

Vrttte(yo) = / g{y{s+ t))d9{s) > [ g{y{Te))d9{s) > {l-y{Ts))9{[Te,oo]) >1-e. 

</[Te,+oo) ^[Te,+oo) 


distance of y{t) from 1 



Figure 5.1: The solution y{t) = l + (yo~ l)e * to the dynamic is represented by the thick curve. 
For given e > 0, > 0 is chosen such that \y{T^) — l\=e. 


Consider now any sequence of evaluations {9^)k which does not contain any subsequence satisfying 
the LTC. Under the assumption that the density fgk for each evaluation 9^ is non increasing, we 
show that Part (i) of Theorem \f. 1\ is not valid: V* 7 ^ sup^gjsj inftg]R_|_ 

Indeed, let us take any yo < 1 and suppose that sup^gjsj (yo) = y*iyo), which is 

equal to 1 as was proved. Let (p{k) be a subsequence such that limfc_>,cx 3 infjgR^ ivo) — I- 

(0‘^(^))fc does not satisfy the LTC by assumption, so Remark (a) implies that there exists 
some T > 0 with 9'^^^\[0,T]) ^ 0. Let iprn be the subsequence of ip and y > 0 such that 
0‘^™-(*^)([O, T]) - > rj. We obtain for any k>l, 

k^oo 

^T 49 vm(k){yo) <Vg^^(k){yo) = [ y(y(t))d(9‘^”"(^^(t) + / y (y(i)) 

*e®+ J[o,T] ^[T,+oo] 

< y(r)0^'=(™)([o,T]) + r''(™)([r,oo]). 

This implies that for such fixed yo < 1 , limfc inftgR_^ ^Ttprm(k) (yo) < y{T)rj + (1 — y) < 1. This 
contradicts the assumption that sup^gj^inftgR_|_ Vlj-^^gk{yo) = 1, and our claim is proved. 


In the second example, we study the convergence of the value functions of a control problem 
along two different sequences of evaluations satisfying the LTC. Along the first sequence, the 
value functions converge uniformly to V*; while along the second, the value functions point- 
wisely converge, but not uniformly (thus the family of value functions is not totally bounded for 
the uniform norm), to a limit function which is different from V*. 


Counter-example 4.11 Consider the control problem on the state space M = (— 00 , + 00 ), where 


12 






the control set is U = {+1,-1}; the dynamic i^: 


f{y, u) = u for all (y, u) E M+ x U and f{y, u) = —1 for all (y, u) E Ml x U, 

where Ml = M_/{0}; and the running cost function is: 

+1 if u = +1, y > 0 
g{y,u) = <( 0 if u = -1, y >0 

+K if y < 0 


Suppose that K > 1 large enough, so the cost on M_ is positive and high. Whenever the state 
reaches y = 0, it is optimal to choose control u = +1 and this drives the state back to M_|_; on 
Ml, the dynamic is f = —1, independent of control and state. Ve{yo) = K for all yo in Ml and 
6 in A(M_|_), so the reduced state space is M_|_, and we consider value functions defined on it. 

y*{yo) = sup 0 inft>o T^ 7 ^tt 6 )(yo) = 0 for any yo > 0. Fix any yo > 0. For any 9 E A(M+) and 
e > 0, let F > 0 such that 0([O, i^]) > 1 — e. Define now the control to he: u'^{t) = +1, if 
t E [0,F] and u^{t) = —1 if t G {F,oo), which gives: 'yTteieiuojU^) < 

Consider {9^)k sequence of evaluations with density fQk{s) = ■^l[fc, 2 fc]('S) for each k, and {9^)k 
the sequence of k-horizon evaluations with density f()k{s) = ^l[o,fc](s) for each k. We show that: 

({Vgfe}, II • ||oo) is totally hounded and (V^fe) converges uniformly to V*; while {{V^k}, || • ||oo) 
is not totally hounded and (V^k) does not converge to V*. 

Let yo > 0, we have that: 


1. V0fc(yo) = 0, for all k > 1. Indeed, one optimal control for VQk{yQ) can he taken as: 
u*{t) = +1, t E [0, k] and u*{t) = —1, t E {k, 2k]; 

2. \Gk{yo) = 0 if k < yo and lAfe(yo) = ^ ~ ^ > 2/o- Indeed, for k < yo, one optimal 

control for \Fk{yo) can be taken as: u*{t) = —1, t E [0, A;]; for k > yo, one optimal control 

for lAfc(yo) can be taken as: u*{t) = +1, t E [0, and u*{t) = —1, t E (, k], so 

lnk{yO,U ) - U - 2 2k- 


See the following two pictures for illustration. 

^Notice that the dynamic is discontinuous at y = 0 when u = +1. To get the desired asymptotic result under 
the Liptchitz regularity, one can slightly modify dynamic to set f(y, +1) = y for y £ [0,1] and others unchanged. 
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Figure 5.2: The left figure describes the dynamic of one optimal control for the evaluation 9^, 
which is u* = +1 on [0, k] and u* = —1 on (k, 2 k]; the right figure describes the dynamic of one 
optimal control for the evaluation 6 ^ with k > yo, which is u* = +1 on [0, ^~ 2 ° ] o-nd u* = —1 on 
(, k]. Here, the vertical axis represents the distance of y{t) from zero and the thick trajectory 
(resp. thin trajectory) corresponds to state on which u = +1 and g = 1 (resp. u = —1 and g = 0 ). 

We deduce that {V 0 k{yo))i. converges uniformly to V*{yo) = 0 on M_|_; and that V-k{yo) - > i, 

“ fc—>-oo 

while the convergence is not uniformly in yo £ indeed, for all k >1, \Lk{k) = 0. 


5 Proof of main result: Theorem 14.11 


Consider in this section a sequence of evaluations {9^)k that satisfies the LTC. As the proof is 
rather long, we divide it into two main parts: 


• in Subsection I5.1[ we present the first preliminary result, Proposition 15.11 It is used in 

two ways: first, we obtain an immediate consequence of it for later use, which bounds 
liminffc Vgk from below in terms of the auxiliary value functions : A; G N*,t E 1^+}; 

second, we deduce from it in Corollary 15.21 the proof for Part (i) of Theorem 14.11 

• In Subsection 15.21 we prove Parts (ii)-(iii) of Theorem 14.11 Lemma 15.41 gives an upper 
bound of limsup^ Vgk in terms of the auxiliary value functions {Vj-^pk : /c € N*,t E ffi+}, 
which is, together with the result from Proposition 15.11 used to end the proof. 


5.1 A first preliminary result and proof for Part (i) 
Proposition 5.1 For any g in A(M_|_), and any initial state yo in 

inf Vrttte( 2 /o) < liminf C 0 ;=(yo)- 

k 
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In particular, we have for all yo in 


sup inf Vj-^ugk{yo) < lira inf Vgk{yo). 


Proof: Fixing y^ and y, we set /3 =def inftgK.,. yTtiBiUo)- For any e > 0 fixed, there exists some 
To > 0 such that ;u([To,oo)) < e. Take any control n(-) in U. By definition of /3, we have that 

VT > 0, [ g{y{t + T,u,yo),u{t + T))dy{t) > 13, 

J [0,+oo) 

thus 

VT > 0, [ g{y{t + T,u,yo),u(t + T))dfi(t)>l3-s. (5.1) 

■l[0,To] 

For each A: > 1, integrating both sides of (I5.ip over T E [0, oo) w.r.t. the evaluation 9^, we obtain 

/ / 9 {y{t + T,u,yo),u{t + T))dg{t)d9^{T) > P - e. (5.2) 

J[0,+oo) J[0,To] 

Applying Fubini’s Theorem to (|5.2I) yields 


/3-e< 


'[OTo] 


' [0,+oo) 


9 {y{t + T, u, yo),u{t + T)) d6i''(T) 


dKt)= [yTtiekiyo,u)]dg{t), 


'[OTo] 


(5.3) 


where 7rttt0‘^(2/O) ■“) = /[o,+oo) v(y(A + T,u,yo),u{t + T))d6»*'(T). According to Lemma [321 we 
have \ jQk{yo,u) — (yO)'“)! < 2TVt{6^). This enables us to rewrite (15.3p as: 

f3-e < f (-fQk{yo,u)+ 2TVt{9^)) dg{t) 

^[0,To] ^ ^ 

< (70.(yo,^) + 2TFTo(0"))M[O,ro]) 

< jgk{yo,u) + 2TVTo{e’^). 

The control u{-) E W being taken arbitrarily, we deduce that 

l3-e<Vgk{yo) + 2TVTo{e^). 

Since (9^) satisfies the LTC, TVto{9^) vanishes as k tends to infinity. The proof is achieved. □ 
We end the proof for Part (i) of Theorem 14.11 by the following corollary of Proposition 15.11 


Corollary 5.2 [Proof for Part (i) of Theorem Ezy 


sup inf PrtttM(yo) = sup inf V-j-^pk{yo), Vyo G 

/iGA(R+) A:>1 *€K+ 
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Proof: Fix yo £ and denote q = sup;i.>]^ inft>o (yo)- It is clear that g < 

inft>o V^((^(yo)- Now for the converse inequality, consider for each k > 1, there 
exists m{k) in M+ such that ivo) ^ 1? + ^/k. Since Tm{k)W^ ~ image measure of 9^ 

by the function s s + m{k) - is also an evaluation on M+, we have: 


Vs > 0, TVs{T^^k)W^) = sup 

QeZ?(]R+) 


e’^{{Q-m{k))rsR+)-e^{{Q-m{k)+s)rm+) < TVs{e^)+e’^{[o,s]) 


We deduce that (7^(fc)tt^^)fc satisfies the LTC whenever {9^)k does so. According to Proposition 
EH V/x E A(M+), inftgM,^ Vr 4 ^{yo) < liminffc (yo) < g, thus sup^gA(R+) infiGR+ yTtiiiivo) < 

g. The proof is complete. □ 


5.2 Proof for Parts (ii)-(iii) 

In this subsection, we give the proof for Parts (ii)-(iii) of Theorem 14.11 We begin with the 
following result, which compares the values under evaluation y and its t-"shifted" evaluation 
Tt^y for any t > 0. 

Lemma 5.3 Let y in A(M_|_) be any evaluation. Then: for all t > 0 and yo E 

Vuivo) - yT4u(yo) + 2rV't(y). 

Proof: Fix y E A(M_|_), t > 0, yQ & M'^. By Lemma EH we have 

Jfj,{yo, u) < 77^j^(yo, u) + 2TVt{y), Vm(-) E U. 

For all e > 0, take E be an e-optimal control for V 77 (t^(yo), be., 77^tj/i(yoj u^) < ^ 771 x 1 ( 2 / 0 ) + 
e. We obtain that 

7m(2/0, u^) < VTtiiiiyo) + £ + 2rPt(y). 

Since P^(yo) = infu(.)gw 7At(2/0 )and e > 0 being arbitrary, we deduce that 

yfliyo) < yTtiuiyo) + 2TV't(y), 


which proves the lemma. □ 

The following result gives an upper bound on "limsup^V^fe" in terms of the auxiliary value 
functions : k E N*,t E M_|_}. 

Lemma 5.4 For all Tq > 0 and any yo in 

limsupP0fc(yo) = limsup inf Pyttefc(yo). 
k k i<T) 

In particular, for all Tq > 0 and any yg in 

limsupV'0fe(yo) < sup inf Pytte'^Cvo)- 

k ken* 
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Proof: Fix Tq > 0 and yo G The inequality " limsup;;. inf^^rp Vj-^pk < limsup^, is clear by 
taking t = 0 for each k. Now for the converse inequality " limsup^ infi<r(, V-j-^^Qk > limsup^ 
according to Proposition I5.3[ we have that for all k and t <Tq, 

Vgkivo) <Vr,pkiyo) + 2TVtie'^). 

For each A; > 1, take < Tq with Vj'^^^Qk{yo) < info<f<To + i: which gives us: 

Vek (yo) < g(l/o) + ^ + 2TV^k {9^) 

~ o<f<ro ^TPto { o '")- 


Since {9^)^ satisfies the LTC, TVtq{9^) vanishes as k tends to infinity. By taking "limsup;," on 
both sides of above inequality, the proof of the lemma is complete. □ 

Now we end the proof for Theorem 14.11 To do this, we first summarize results in Proposition 15.II 
and Lemma [5.41 in the following chain form, which is then used for the study of the convergence 
of iVgk)k. 


Corollary 5.5 For all Tq > 0 and yo in 

sup inf Vj-Mkiyo) > limsup ^^^(yo) > lim inf f 4 »=(yo) > sup inf F 7 ^[( 0 fe(yo) 
k>l i<T) k k>i *>o 

Remark 5.6 Corollary 15.51 states that the uniform convergence of "supi^yi inft<'r(j y-ji^gk " to 
"supj^^i init>o ^0 tends to infinity implies the uniform convergence of {Vgk)k as k 

tends to infinity. Moreover, according to Corollarv \5.Sl in case of uniform convergence, the limit 
function is V*. 


For any states y and y in M'^, let us define d{y,y) = sup;i.>i |V' 0 fc(y) — Vgk{y)\. The space {W^,d) 
is now a pseudometric space (may not be Hausdorff). 

The following is similar to the proof of Theorem 2.5 in |9], and is also similar to the proof of 
Theorem 3.10 in |10| . We rewrite it here for sake of completeness. Roughly speaking, we shall 
use the total boundedness of the space {{Vgk}, || • ||oo) so as to deduce that the state space (M'^, d) 
is totally bounded for the pseudometric metric d. This allows us to prove the convergence for d of 
the reachable set to R in bounded time. We are then able to prove the uniform convergence 
of "supfc>i infi<ro Vrttte" to "supfc>i infi>o VVttte" as Tq tends to infinity.. 

Proof for Theorem 14.11 Parts (ii)-(iii). 

We first prove Part (iii). One direction is clear: the uniform convergence of {Vgk) implies the 
totally boundedness of the space {{Vgk}, || • ||oo)- 

Let us prove the converse. Suppose that {{Vgk}, || • | |oo) is totally bounded, so fixing any e > 0, 
there exists a finite set of indices I such that for all fc > 1, there exists i ^ I satisfying 

\\Vgk — Vgilloo < e/3. 
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is a subset of the compact metric space ([ 0 , 1 ]^, || • ||oo)) thus it is itself totally 
bounded, so there exists a finite subset X of such that 

Vy G E X,Vz E /, \Vf)i{y) - Vsi{x)\ < e/3. 

We have obtained that for each e > 0, there exists a finite subset X of such that for every 
y E there is x E X satisfying: for any k > 1 there is some f E / with 

\Vg,{y) - Vg,ix)\ < \Vg,iy) - Vg.iy)\ + \VM - VAx)\ + \Vg.{x) - Vg,{x)\ < S, 

thus d{y, x) < e. This implies that the pseudometric space (M'^, d) is itself totally bounded. 

Fix now yo in It is by definition that 

for all T, 5 E M+ with S >T, we have fF{ yo) C R^{yo) C R{yo), 

and 

Vy E R{yo), 3T > 0 with y E R^{yo)- 

From the totally boundedness of we show that converges to R in the following 

sense 

Ve > 0, 3r > 0 ; Vy E R{y), 3y E i?'^(y), d{y,y) < £■ (5.4) 

Indeed, let us first take {y^} a finite e-cover of R{yo) for d. For each yi, put > 0 with 
yi E R'^^{yo). We then take T = maxT^. Now for any y E R{yo), there is some yi with 
d{y,yd < £• Moreover, y^ E R^^{yo) C i?^(yo). This proves (I5.4jl . 

By Corollary ESI for all T > 0 ( using Ve’^iy) = inft<r (vo), cf. dM])): 

sup inf V 0 k{y) > hmsupl/ 0 fc(yo) > liminf V))fe(yo) > sup_ inf Vgk{y). 
k>iy&R'^(yo) k k k>iy&R{yo) 

Consider fc > 1 and T > 0 given by assertion (15.4j) for the fixed e > 0. Let y E R{yo) be such that 
V^ki^o) < inf^g^(y) VQk(jj) + £, and then y in R^(yo) be such that d{y,y) < e. Since V^k is clearly 
1-Lipschitz for d, we obtain VQk{y) < inf^^g^j-yQ) Vgk{y) + 2s. Consequently, inf^g^T(^j,) he'=(y) ^ 
in%Gij(yo) for all k, so 

sup inf V 0 k{y) < sup_ inf V 0 k{y) + 2 s. 
k>iy&R'^{yo) k>iy<£R(yo) 

One obtains that limsupfc>]^ bgfc(yo) < liminffc>i (yo)-|- 2 e, and so {V 0 k{yo)'jj^ converges. Since 
(M'^, d) is totally bounded and all Vgfe are 1-Liptschitz, the convergence is uniform. 

Next, Part (ii) can be deduced from the proof of Part (iii). Let be any subsequence 

of ( 6 ^) that converges uniformly to some function V. This implies that ({V))^ 5 (fe)}, || • ||oo) is 
totally bounded. As we have shown in the proof of Part (iii) that if {{V 0 ^(k)^, || • ||oo) is totally 
bounded, {V 0 ^(k)) converges uniformly to C = V*, which implies Part (ii) that V* is the unique 
accumulation point (for the uniform convergence) of the sequence (Vgfe)^. □ 


18 


6 Discussion on a weaker long-term condition 


Below is a weaker form of the long-term condition (LTC): 

Long-term condition’ (LTC’) A sequence of evaluations {9^)k>i satisfies the LTC’ if: 

Vs > 0, TVs{e’^) -^ 0. (6.1) 

/c—>-oo 


It is unclear whether the LTC’ is strictly weaker than the LTC or not. One might want to 
construct an example of {0^)k such that TVs{9^) -)• 0 for all s > 0 while TVsq{9^) -^ 

/c—>-oo /c—>-oo 

a > 0 for some sq > 0 and a > 0. The following example shows that this is possible if we 
consider only s being rational numbers. In general, the question is still open. 


Example 6.1 Given a positive integer k, consider the density 9^ with support included in [0, A:] 
by dividing [0, k] in k‘^ consecutive small intervals of length 1/k, and 9^ is uniform over the union 
of all small odd intervals... and puts no weight on even small intervals. Define the support 


Sk 


u 


21 2l + l\ 

k J ■ 


9^ has density: 

2 2 

fk{x) = IxSSj, — llxG[0,fc],E(fea;)g2N 

(where 2N is the set of even numbers in N, E(x) is the integer part of x). 
For each k, we have (consider s = l/fc); 


sup [ \fkix +s) - fk{x)\dx>2-l/k 

0<S<1 Jx>0 

Consider now only k of the form n\, and we define the density gn = fn\ for each n in N. For all 
X > 0, 

2 

9n{,X T s) 9 n{,x) ~ (li?(n!(3;+s))£2N,3;-|-s<n! k£;(^!3,)g2N,a;<n!) • 

Assume s is a rational number. Then for n large enough, nls is an even integer, so for all x 
such that 0 < X < n\ — s, we have Qnix -|- s) — gn{x) = 0. Consequently, 



\gn{x + s) - gn{x)\dx 


-> 0 . 

n—>-oo 


7 Appendix 


Proof for Lemma [T) The following computation of /^(t) is straightforward: 

1 


Vt > 0, f'f){t) = 


a 




exp 


^ ft — m — t 


a 




exp 


1 m + t 

2 \~^) / 
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thus 

/g(t) > 0 {resp. < 0) (m—t)exp (m+t)exp ^ ® (resp. < 0). 

As a consequence, one obtains that /^(t) <0, Vt > m. Now we look at t G (0,m). Denote 
H{t) =def exp which enables us to write: 

fg{t) > 0 {resp. < 0) H{t) >0 {resp. < 0), Vt G (0,m). 

From the above analysis, we deduce that the proof of the lemma is reduced to the proof for 

Claim There is some t* G [0,m) such that H{t) < 0 fort G {0,t*) and H{t) > 0 fort G {t*,m). 
Moreover, such t* satisfies (t*)^ > 

In order to prove the claim, we compute 


the values at the end point: H{0) =0 and limj_,.^- H{t) = —oo; 

the first-order derivative at any t G [0, m): 

,, , / 2mt\ 2m 2m 

H'{t) = exp ^ ^ - 


cr^ 


cr^ 


(m — ty 


(7.1) 


• at any rest point t^ G [0,m) {i.e., H{t^) = 0): 

/ 2mt^ \ m + t^ 
exp —^ 

\ J m — r 

which is substituted back into (EU, to yield 

H'{t^) > 0 ( resp. H'{t^) < 0 ) ( resp. > mf — a'^ ). (7.2) 


Next, it is easy for us to prove the following result: 

Let t\ G [0,m) he a rest point for H{-), and suppose that t^ G {t\,m) is the smallest rest point 
after t\. Then H’{t\)H'{t^ < 0 and if Hft^) < 0, such does not exist. 

Indeed, H'{t\)H'{ff) < 0 can be derived from the continuity of H{-)] suppose that H'{tf) < 0, 
we have from (j7.2jl that (tf)^ > m? — and H'{t\) > 0, thus (t|)^ < mf — cr^. However, this 
leads to a controdiction to > tf, so does not exist whenever H'{tf) < 0. 

Finally, remark that 77(0) = 0, thus t = 0 is a rest point. We discuss the following two cases: 

Case 1. m? — < 0, thus 77'(0) < 0. 

This implies that no rest point exists after 0. Since lim^^^- H{t) = —oo, we deduce that 
H{t) <0, Vt G (0,m). The claim is proved for t* = 0. 

Case 2. mf — > 0, thus 77'(0) > 0. 

limj_j.^- H{t) = —oo implies that some rest point exists in (0, m). Take the closest to 0, 
implying that H{t) >0, Vt G (0, t*^). Further, we obtain that H'{t^) < 0 by the continuity of 
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H{-). Again, there exists no other rest point after Since limj_j.^- H{t) = —oo, we deduce 
that H{t) <0, Vt G The claim is proved for t* = 

To conclude, we see that in both cases such t* exists and satisfies — ci^, thus the 

claim is proved. This finishes our proof for the lemma. □ 
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